Backports for 1.12.0-alpha1 #57258
Backports for 1.12.0-alpha1 #57258
Conversation
I couldn't find `time_ns` when I was looking for it, nice to make clear the "monotonic" clock is also available in Base. (cherry picked from commit a52de83)
|
@nanosoldier |
|
@KristofferC Should this PR target |
|
Oops, yes, thanks! |
|
I think I'll have to make a commit to this branch just for the sake of kicking off 1.12 Buildkite for the first time - I'll immediately revert the commit, and from then on CI should work correctly. |
|
Looks like the 1.12 Buildkite CI has now been triggered correctly on this PR. |
|
Alright, looks like the 1.12 Buildkite pipeline is working. |
…57241) The `fork()` we do here relies on `SIGCHLD` to make sure that we don't race against the child. This is easy to see in an embedding application that dynamically links `libjulia`: ```c int main(int argc, char *argv[]) { signal(SIGCHLD, SIG_IGN); void *handle = dlopen("path/to/libjulia.so", RTLD_LAZY); return 0; } ``` Without this change, this fails with an error message: ``` Error during libstdcxxprobe in parent process: waitpid: No child processes ``` Resolves #57240 (cherry picked from commit daf865e)
Restores #57035, undo #57089 for non-FreeBSD. While I suggested doing this change for all platforms, I forgot that means non-FreeBSD platforms become vulnerable again to the very deadlock problems that #57035 was required to prevent. That fix seems to not be viable on FreeBSD due to known libc implementation problems on that platform. However, upon closer inspection of the questionable design implementation decisions they seem to have made here, the platform is likely not currently vulnerable to this libunwind bug in the first place: https://github.com/lattera/freebsd/blob/master/libexec/rtld-elf/rtld_lock.c#L120 (cherry picked from commit 2f0a523)
It looks like these methods were just missed while overloading for BufferStream. There's also `readbytes!` where the current implementation will fallback to the `LibuvStream` implementation that is currently not threadsafe. What's the best approach there since the implementation is quite a bit more involved? Just duplicate the code but for BufferStream? Should we take the BufferStream lock and invoke the LibuvStream method? Open to ideas there. Also open to suggestions for having tests here? Not easy to simulate the data race of writing and calling readavailable. The fix here will unblock JuliaWeb/HTTP.jl#1213 (I'll probably do some compat shim there until this is fully released). Thanks to @oscardssmith for rubber ducking this issue with me. Probably most helpfully reviewed by @vtjnash. --------- Co-authored-by: Jameson Nash <[email protected]> (cherry picked from commit ffc96bc)
(cherry picked from commit 99fd5d9)
This is the final PR in the binding partitions series (modulo bugs and tweaks), i.e. it closes #54654 and thus closes #40399, which was the original design sketch. This thus activates the full designed semantics for binding partitions, in particular allowing safe replacement of const bindings. It in particular allows struct redefinitions. This thus closes timholy/Revise.jl#18 and also closes #38584. The biggest semantic change here is probably that this gets rid of the notion of "resolvedness" of a binding. Previously, a lot of the behavior of our implementation depended on when bindings were "resolved", which could happen at basically an arbitrary point (in the compiler, in REPL completion, in a different thread), making a lot of the semantics around bindings ill- or at least implementation-defined. There are several related issues in the bugtracker, so this closes #14055 closes #44604 closes #46354 closes #30277 It is also the last step to close #24569. It also supports bindings for undef->defined transitions and thus closes #53958 closes #54733 - however, this is not activated yet for performance reasons and may need some further optimization. Since resolvedness no longer exists, we need to replace it with some hopefully more well-defined semantics. I will describe the semantics below, but before I do I will make two notes: 1. There are a number of cases where these semantics will behave slightly differently than the old semantics absent some other task going around resolving random bindings. 2. The new behavior (except for the replacement stuff) was generally permissible under the old semantics if the bindings happened to be resolved at the right time. With all that said, there are essentially three "strengths" of bindings: 1. Implicit Bindings: Anything implicitly obtained from `using Mod`, "no binding", plus slightly more exotic corner cases around conflicts 2. Weakly declared bindings: Declared using `global sym` and nothing else 3. Strongly declared bindings: Declared using `global sym::T`, `const sym=val`, `import Mod: sym`, `using Mod: sym` or as an implicit strong global declaration in `sym=val`, where `sym` is known to be global (either by being at toplevle or as `global sym=val` inside a function). In general, you always allowed to syntactically replace a weaker binding by a stronger one (although the runtime permits arbitrary binding deletion now, this is just a syntactic constraint to catch errors). Second, any implicit binding can be replaced by other implicit bindings as the result of changing the `using`'ed module. And lastly, any constants may be replaced by any other constants (irrespective of type). We do not currently allow replacing globals, but may consider changing that in 1.13. This is mostly how things used to work, as well in the absence of any stray external binding resolutions. The most prominent difference is probably this one: ``` set_foo!() = global foo = 1 ``` In the above terminology, this now always declares a "strongly declared binding", whereas before it declared a "weakly declared binding" that would become strongly declared on first write to the global (unless of course somebody had created a different strongly declared global in the meantime). To see the difference, this is now disallowed: ``` julia> set_foo!() = global foo = 1 set_foo! (generic function with 1 method) julia> const foo = 1 ERROR: cannot declare Main.foo constant; it was already declared global Stacktrace: [1] top-level scope @ REPL[2]:1 ``` Before it would depend on the order of binding resolution (although it just crashes on current master for some reason - whoops, probably my fault). Another major change is the ambiguousness of imports. In: ``` module M1; export x; x=1; end module M2; export x; x=2; end using .M1, .M2 ``` the binding `Main.x` is now always ambiguous and will throw on access. Before which binding you get, would depend on resolution order. To choose one, use an explicit import (which was the behavior you would previously get if neither binding was resolved before both imports). (cherry picked from commit 888cf03)
KristofferC
force-pushed
the
backports-release-1.12
branch
from
4bb5cb3 to
ae78058
Compare
|
The package evaluation job you requested has completed - possible new issues were detected. Report summary❗ Packages that crashed168 packages crashed only on the current version.
4 packages crashed on the previous version too. ✖ Packages that failed1421 packages failed only on the current version.
2709 packages failed on the previous version too. ✔ Packages that passed tests36 packages passed tests only on the current version.
4580 packages passed tests on the previous version too. ➖ Packages that were skipped altogether61 packages were skipped only on the current version.
1300 packages were skipped on the previous version too. |
Fixes #56864. Ref https://github.com/JuliaLang/julia/blob/99fd5d9a92190e826bc462d5739e7be948a3bf44/stdlib/REPL/src/LineEdit.jl#L504 --------- Co-authored-by: Ian Butterworth <[email protected]> (cherry picked from commit f0446c6)
This comment was marked as outdated.
This comment was marked as outdated.
|
The package evaluation job you requested has completed - possible new issues were detected. Report summary❗ Packages that crashed92 packages crashed only on the current version.
✖ Packages that failed1032 packages failed only on the current version.
81 packages failed on the previous version too. ✔ Packages that passed tests292 packages passed tests on the previous version too. |
|
|
Addresses review comment in #57212 (comment). The key is that the hand-off of responsibility for verification between the loading code and the ordinary backedge mechanism happens under the world counter lock to ensure synchronization. (cherry picked from commit 34aceb5)
Updating mmtk-julia version to include mmtk/mmtk-julia#228 and fix the MMTk CI. I've also changed the allocation profiler tests to skip all tests instead of just a few since I've seen some spurious errors - they should all be related though, we need to make sure the profiler accounts for fastpath allocation (see #57103) This should fix #57306. (cherry picked from commit 72f8a10)
Similar to #57229, this commit ensures that `Compiler.finish!` properly synchronizes the operations to set `max_world` for cached `CodeInstance`s by holding the world counter lock. Previously, `Compiler.finish!` relied on a narrow timing window to avoid race conditions, which was not a robust approach in a concurrent execution environment. This change ensures that `Compiler.finish!` holds the appropriate lock (via `jl_promote_ci_to_current`). (cherry picked from commit 4ebb50b)
(cherry picked from commit 132057c)
Fixes #54560 (comment) (cherry picked from commit 79e98e3)
This adds a warning for the auto-import of types cases (#25744) that we have long considered a bit of a bug, but didn't want to change because it is too breaking. The reason to do it now is that the binding rework has made this case more problematic (see #57290). To summarize, the question is what happens when the compiler sees `f(x) = ...` and `f` is currently and implicitly imported binding. There are two options: 1. We add a method to the generic function referred to by `f`, or 2. We create a new generic function `f` in the current module. Historically, case 1 has the additional complication that this error'd unless `f` is a type. It is my impression that a lot of existing code did not have a particularly good understanding of the resolved-ness dependence of this behavior. However, because case 1 errors for generic functions, it appears that existing code generally expects case 2. On the other hand, for types, there is existing code in both directions (#57290 is an example of case 2; see #57302 for examples of case 1). That said, case 1 is more common (because types tend to be resolved because they're used in signatures at toplevel). Thus, to retain compatibility, the current behavior on master (where resolvedness is no longer available) is that we always choose case 2 for functions and case 1 for types. This inconsistency is unfortunate, but I tried resolving this in either way (making all situations case 1 or all case 2) and the result was too breaking. Nevertheless, it is problematic that there is existing code that expects case 2 beavior for types and we should help users to know what the correct way to fix it is. The proposed resolution is thus: 1. Retain case 1 behavior for types 2. Make it a warning to use, encouraging people to explicitly import, since we generally consider the #25744 case a bug. Example: ``` julia> module Foo String(i::Int) = i end WARNING: Type Core.String was auto-`import`ed in `Foo`. NOTE: This behavior is deprecated and may change in future Julia versions. NOTE: This behavior may have differed in Julia versions prior to 1.12 depending on binding resolution. Hint: To retain the current behavior, add an explicit `import Core: String` in Foo. Hint: To create a new generic function of the same name use `function String end`. Main.Foo ``` (cherry picked from commit 8c62f42)
introduced in #56144 --------- Co-authored-by: Mosè Giordano <[email protected]> (cherry picked from commit dd13878)
This should be enough to get rid of the 3 lingering warnings in our main test suite. (cherry picked from commit 751a0d7)
Stdlib: SparseArrays URL: https://github.com/JuliaSparse/SparseArrays.jl.git Stdlib branch: main Julia branch: master Old commit: 212981b New commit: 72c7cac Julia version: 1.13.0-DEV SparseArrays version: 1.12.0(Does not match) Bump invoked by: @ViralBShah Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaSparse/SparseArrays.jl@212981b...72c7cac ``` $ git log --oneline 212981b..72c7cac 72c7cac Explicitly declare type constructor imports (#598) ff083ce README: update github action badge (#596) ``` Co-authored-by: ViralBShah <[email protected]> (cherry picked from commit ac79b9f)
@KristofferC requested that `function @main(args) end` should work. This is currently a parse error. This PR makes it work as expected by allowing macrocall as a valid signature in function (needs to exapand to a call expr). Note that this is only the flisp changes. If this PR is accepted, an equivalent change would need to be made in JuliaSyntax. (cherry picked from commit b65f004)
https://github.com/openssl/openssl/releases/tag/openssl-3.0.16 (cherry picked from commit 7ec984a)
This will cause these values to be populated at slightly different times than now, but I don't think it will matter. (cherry picked from commit 55d5a4b)
This dispatch should not be treated as resolved just because arg0 is constant. With the previous code, that meant it was eligible for last-minute call resolution, but call-resolution in codegen is now forbidden so this needs to fail unilaterally. (cherry picked from commit 0b74d17)
also test that the difference `OncePerX` infer (cherry picked from commit 73506ed)
…`file` metadata (#57359) Based on the discussion in #55963 (comment) (cherry picked from commit c46b533)
This should get fix the binding world-age warnings in our `doctest` CI (cherry picked from commit 796a849)
#55906 changed `cconvert(Ref{BigFloat}, x::BigFloat)` to return `x.d`, but neglected to do so for other types of `x`, where it still returns a `Ref{BigFloat}` and hence is now returning the wrong type for `ccall`. Not only does this break backwards compatibility (JuliaMath/SpecialFunctions.jl#485), but it also seems simply wrong: the *whole* job of `cconvert` is to convert objects to the correct type for use with `ccall`. This PR does so (at least for `Number` and `Ref{BigFloat}`). (cherry picked from commit 6dca4f4)
|
@nanosoldier |
|
Will start merging these a bit more frequently since people can now "subscribe" to the release branch using the |
|
The package evaluation job you requested has completed - possible new issues were detected. Report summary❗ Packages that crashed84 packages crashed only on the current version.
✖ Packages that failed892 packages failed only on the current version.
40 packages failed on the previous version too. ✔ Packages that passed tests96 packages passed tests on the previous version too. |
Backported PRs:
waitpidrace condition whenSIGCHLDis set toSIG_IGN#57241Compiler.finish!#57248OncePerXsubtypeFunction#57289Non-merged PRs with backport label:
Expr(:call, ...)#57342RationalfromAbstractIrrational#55853